Touch responsive displays are used on personal data assistants (PDAs), tablet personal computers (PCs), and as monitors that can be coupled to PCs. However, most such displays are simply pressure activated or employ optical sensors around the periphery of the display surface. Another type of touch sensitive display has been developed in the art that uses optical vision systems for interactively detecting an object touching a display surface. Examples of these are the metaDesk developed by the MIT Media Lab, as reported by Brygg Ullmer and Hiroshi Ishii in “The metaDESK: Models and Prototypes for Tangible User Interfaces,” Proceedings of UIST 10/1997:14-17, and a display surface as disclosed in several papers published by Jun Rekimoto of Sony Computer Science Laboratory, Inc., in collaboration with others.
An interactive display system like these that have previously been developed typically includes a projector for projecting an image on a surface, and a vision system for detecting objects on the surface. Almost no information is provided in references about these interactive display systems about aligning the projected image with the vision systems. A desired interactive experience when using an interactive display system would ensure that the system properly respond to a user touching a virtual (projected) object, such as a virtual control button appearing on the display surface. However, if the position of the virtual control button in the projected image is not properly aligned with the position of the user's finger sensed by the vision system, the interactive display system will not respond correctly. Optical errors in the vision system must be corrected to enable the vision system to provide an accurate determination of the location where an object is touching the display surface. For example, use of a wide angle lens in the vision sensing system can cause barrel distortion. If the sensor used to detect an object on the display surface is mounted off to one side of the surface, its view of the display surface will cause a trapezoidal distortion.
A further source of problems can arise if the illumination used by the vision system to detect objects on the display surface is non-uniform, since an uneven illumination can cause erratic sensing of objects on a display surface if not compensated or corrected. Many pattern recognition techniques employed by a vision system assume that an absolute pixel intensity level is detected by the vision system and will fail if the appearance of an object in the vision system is brighter or dimmer at various different positions of the object on a surface. It is important that the behavior of these type algorithms be independent of the location of an object on a surface. The vision system should also be capable to distinguish when an object is touching a display surface from when an object is simply close to the display surface. Since the skin of different users can differ in reflectivity, it may be necessary to determine a threshold level for the user of the interactive system so that false determination of each user touching the display surface is avoided.
Accordingly, in order for an interactive display system to properly respond to a user touching the display surface, the optics of the system must be aligned and calibrated, various types of distortion must be removed, and compensation must be made for variations between users. Clearly, it would be preferable to perform the calibration and alignment of the interactive system in an automated manner, before the interactive system is shipped from the factory. However, alignment of the projector and vision system can change over time or do to rough handling while the system is being shipped. Accordingly, it would be desirable to enable each interactive display system to be easily aligned after leaving the factory, either by a skilled user or a technician. It would also be desirable to adjust the threshold for user touch on the display surface in an unobtrusive way, so that the user is generally not aware that the process is being carried out.